feat(config): add GLM-5.2 GB300 AgentX concurrency-1 disaggregated point / 添加 GLM-5.2 GB300 AgentX 并发度 1 分离式配置 - #2720
Conversation
新增 GLM-5.2 GB300 AgentX 并发度 1 的预填充与解码分离配置，并移除已替代的聚合配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2720 链接。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Move the GLM-5.2 concurrency-1 AgentX point from an aggregated TP8 topology to a disaggregated 1-prefill/1-decode topology with TP4 attention-DP prefill and TP8 decode." | ||
| - "Use MTP depth 5 with the committed golden acceptance target, tensor-parallel MoE on decode, and the CUTEDSL top-k path." | ||
| - "Align recipe KV offloading with the master config and keep the benchmark client colocated with the frontend orchestrator." | ||
| - "Retire the superseded aggregated configuration key while retaining the existing disaggregated topology points." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
There was a problem hiding this comment.
🟡 The new perf-changelog.yaml entry appended by this PR ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX instead of the actual PR number (2720). Every other entry in the file uses a real PR number, so this leaves a broken/dead link at the end of an append-only, byte-sensitive changelog. Please replace XXX with 2720 before merge.
Extended reasoning...
The last entry in perf-changelog.yaml (lines 6396-6400, appended by this PR) ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXXXX is a literal, unfilled placeholder rather than an actual PR number. This is PR #2720, and the correct value is simply 2720.
Where this comes from: every prior entry in the file uses a real numeric PR link, e.g. the immediately preceding entry ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2650 (line 6390), and other nearby entries reference 2685 and 2707. The new entry added by this PR is the only one with XXX instead of a number, which strongly suggests the author copy-pasted the changelog template and forgot to substitute in the real PR number before committing.
Why nothing catches this automatically: the changelog is append-only and treated as byte-sensitive (the PR checklist explicitly calls out appending a new entry without editing historical ones), but there's no schema validation or CI check enforcing that pr-link resolves to a real, existing PR — a string like .../pull/XXX is syntactically valid YAML and a syntactically valid URL, so nothing fails at parse time or link-check time (XXX would 404 if fetched, but no such link check appears to run in CI for this file).
Concrete proof / step-by-step:
- Open
perf-changelog.yamland go to line 6400 (the last line of the file, added in this diff). - Read the value:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. - Compare to the entry directly above it (line 6390):
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2650— a real, resolvable PR number. - Navigate to
https://github.com/SemiAnalysisAI/InferenceX/pull/XXX— this 404s, whereas.../pull/2650resolves to a real PR. - This PR is feat(config): add GLM-5.2 GB300 AgentX concurrency-1 disaggregated point / 添加 GLM-5.2 GB300 AgentX 并发度 1 分离式配置 #2720 (per the PR metadata), so the correct fix is to replace
XXXwith2720.
Impact: this doesn't affect any runtime or benchmark behavior — it's purely a traceability/documentation defect in the changelog. But because the file is append-only and treated as immutable history once merged, an incorrect placeholder value can't be silently patched later without an explicit follow-up commit calling out the correction, so it's worth fixing now while the entry is still open in this PR.
Fix: change line 6400 from pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2720.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32758589490 |
将 GLM-5.2 扫描临时限定为并发度 1，并同步 main 分支的变更。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32790767048 |
将 GLM-5.2 并发度 1 配置添加到现有曲线，并保留所有已发布点。
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID: 'true' | ||
| AIPERF_USE_DYNAMO_CONV_AWARE_ROUTING: '0' | ||
| OPENAI_API_KEY: EMPTY | ||
| KV_OFFLOADING: dram |
There was a problem hiding this comment.
DRAM offload env vars missing
High Severity
The new recipe sets KV_OFFLOADING to dram in benchmark.env, but omits KV_OFFLOAD_BACKEND and TOTAL_CPU_DRAM_GB. agentic_srt.sh sources benchmark_lib.sh, which exits immediately when DRAM offload is enabled without those variables, so the new concurrency-1 point fails before replay starts. Sibling GLM-5.2 srt-slurm recipes keep KV_OFFLOADING as none and do not take this path.
Reviewed by Cursor Bugbot for commit 5e7d158. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32894023218 |
恢复 GLM-5.2 配置的完整七点扫描覆盖。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33004185970 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33004185970 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33004185970 |
# Conflicts: # perf-changelog.yaml
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9a3e9c4. Configure here.
| - "Add HiCache host-DRAM KV tier arms at TP4 concurrency 40, 48, 56, and 64 and TP2 concurrency 20, 24, 28, and 32, using hicache ratio 1.5 with write_through, direct io, and page_first_direct layout." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2693 | ||
|
|
||
|
|
There was a problem hiding this comment.
Changelog historical bytes were edited
Medium Severity
A blank separator line was deleted between existing perf-changelog.yaml entries instead of appending only at the tail. This breaks the append-only, byte-sensitive changelog rule, so the raw historical-prefix check will reject the change and changelog merge tooling that depends on an exact prefix can fail.
Reviewed by Cursor Bugbot for commit 9a3e9c4. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33098715713 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33098715713 |
2 participants
Description
Add a GLM-5.2 GB300 AgentX concurrency-1 point to the existing disaggregated configuration using a 1-prefill/1-decode topology. The aggregated concurrency-1 point and all six existing disaggregated points remain unchanged.
The new recipe uses TP4 attention-DP prefill and TP8 decode, MTP depth 5 with the committed golden acceptance target, tensor-parallel MoE on decode, CUTEDSL top-k, and DRAM KV offloading. The regular changelog entry selects the complete seven-point disaggregated configuration and its applicable existing eval leaves for the final full sweep.
中文说明
将 GLM-5.2 GB300 AgentX 并发度 1 的新点添加到现有分离式配置中，采用 1 个预填充工作进程和 1 个解码工作进程的拓扑。原有聚合式并发度 1 点以及现有 6 个分离式点均保持不变。
新配置使用 TP4 注意力数据并行预填充和 TP8 解码、MTP 深度 5 及已提交的黄金接受长度目标、解码侧张量并行 MoE、CUTEDSL top-k 和 DRAM KV 缓存卸载。常规性能变更日志条目会在最终完整扫描中选择全部 7 个分离式点及其适用的现有评测任务。
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.Note
Low Risk
Benchmark and recipe configuration only; no runtime or auth changes, and existing published disagg points are retained.
Overview
Adds a concurrency-1 AgentX point to the existing GLM-5.2 NVFP4 GB300 Dynamo + TensorRT-LLM disaggregated sweep, bringing the documented topology count from six to seven while leaving the aggregated c1 point and other disagg arms unchanged.
Introduces
dynamo-disagg-gb300-1p1d-tp8-c1-b1-mtp5.yamlfor 1 prefill / 1 decode serving: TP4 attention-DP prefill, TP8 decode, NIXL KV transfer, MTP depth 5 with the golden acceptance env override, decode tensor-parallel MoE with CUTEDSL sparse top-k on prefill and CuteDSL paths on decode, and DRAM KV offloading aligned with the master scenario. Wires it intoglm5.2-fp4-gb300-dynamo-trt-agentic-disagg-mtpunderagentic-codingwithconc-list: [1]and records the change inperf-changelog.yaml.Reviewed by Cursor Bugbot for commit 9a3e9c4. Bugbot is set up for automated code reviews on this repo. Configure here.